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ABSTRACT 

There may be some important unintended consequences if states 
do not go beyond the "adequate yearly progress" (AYP) requirement of the No 
Child Left Behind Act. Analyses of data from states in which accountability 
systems in place have been based on the percentage of students reaching a 
certain proficiency level suggest that many educators have responded to the 
pressures of raising the percent proficient by choosing to focus their 
instruction on students closest to meeting the proficiency standards. In the 
long run, such a strategy may be a detriment to meeting AYP in future grades. 
Students at the lower end of the achievement spectrum may fall further 
behind; higher achieving students who experience suppressed growth will 
profile closer to the proficient/nonproficient cut point. This is but one of 
the many reasons states should go beyond AYP to set a growth standard with a 
value-added assessment system. The paper outlines test data requirements for 
a robust value-added assessment system and statistical criteria for such a 
system. (SLD) 
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Beyond No Child Left Behind 



William L. Sanders, Ph.D. 1 

Clearly, the intent of the recently enacted No Child Left Behind(NCLB) federal legislation is to set 
an academic floor for America’s student population. Its enactment has certainly changed the 
national landscape relative to the testing of America’s students. All students in all states will be 
tested annually in math and reading for grades 3-8. Each of these tests is to be linked with a 
state’s curricular standards, along with an accompanying definition of proficiency level attached 
for each grade and subject. Each district and school must make adequate yearly progress (AYP) 
with all students collectively and with each identifiable subgroup of students (explicitly defined by 
the U.S. Department of Education), or serious sanctions can occur. The federal definition of 
AYP is not a value-added measure of progress for individual students, but rather it requires cross- 
cohort comparisons of the percent of students meeting the proficiency standards this year 
compared to last year’s percent proficient. If this percentage of proficient students for a district 
or a school is not sufficiently greater than last year’s percentage, then this school or district will 
have failed to meet the AYP requirement. Additionally, each identified subpopulation considered 
to be at risk for academic failure must meet a similar requirement. Presently, states have no 
latitude as to how AYP is to be calculated. 

However, there may be some important unintended consequences of this legislation if states do 
not go beyond the No Child Left Behind AYP requirement. We have had the opportunity to 
analyze several datasets, coming from districts within states that have had an accountability 
system in place that has been based primarily upon the percentage of students reaching a certain 
proficiency level. Especially for schools serving disadvantaged populations of students (e.g., low 
SES), we have observed too often that students whose achievement was above the proficiency 
level had suppressed academic gains. 

It is our interpretation from these analyses that many educators have responded to the pressures of 
raising the percent proficient by choosing to focus their instruction primarily for those students 
closest to meeting the proficiency standards. In the short run by restricting the focus to students 
perceived to be near proficient, while overlooking those who are very low or high achieving, this 
strategy (consciously or sub-consciously adopted) may result in increasing the percent proficient 
in the short term, but in the longer run may be a detriment to meeting AYP in future grades. Not 
only will those students at the lower end of the achievement spectrum fall farther behind, but also 
the higher achieving students who consistently experience suppressed growth will profile closer to 
the proficient/non-proficient cut, decreasing their probability of demonstrating proficiency on a 
subsequent academic milestone. 

To the extent that this practice becomes prevalent, it could be especially harmful to early high 
achieving students from at-risk populations. These high achieving students tend to be in the 
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minority in their low achieving schools. Teachers who perceive these early high achievers to be at 
minimal risk for meeting the most eminent achievement bar may focus most of their effort on the 
lower achieving students under their tutelage. These teachers may fail to recognize that without 
appropriate academic progress each year, the higher achieving students may become at risk for 
meeting future achievement requirements. Thus, if the only feedback to educators and the public 
from the annual test data is merely the pass/fail results of AYP and if educators succumb to the 
pressures described above, there is potential for encouraging practice that will suppress the 
growth of many student populations. This suppressed growth will cause the students’ ultimate 
attainment to be at a lower level rather than a higher one at the culmination of their K-12 
experience. This, indeed, is not the intent of NCLB, but it will become reality in many places. 

Our research has documented the necessity of appropriate progress each year if students are to 
leave their K-12 experience sufficiently prepared for employment or college success. A subtle 
suppression of student growth, a little at a time, year by year, will result in a smaller percentage 
of students belonging to an at-risk subpopulation reaching existing higher levels of academic 
achievement. The outcome for these students will be that they may be less well prepared for 
either employment or college work than their peers of comparable previous achievement who 
graduated prior to the enactment of the legislation. Unfortunately, reporting the number of 
students at proficiency or above will not detect this problem. 

In our view this is but one of many reasons that states should go beyond the reporting 
requirements of NCLB and include in their accountability system a value-added (growth) 
dimension. Since each state will be required to test each student each year in math and reading 
(grades 3-8), the data will soon be available for each state to monitor the academic progress of 
each student over years. From this longitudinally merged data structure, an appropriate value- 
added assessment system can provide estimates of the impact of educational entities on the rate of 
progress of various sub-groups of students. By setting a growth standard, then “all students will 
count” and some of the perverse unintended consequences will be dampened. 

What is meant by value-added analysis? The usage of the term “value-added” as applied in 
educational assessment has become rather commonplace in recent years, but this term does not 
imply, innately, specific definitions or indications of either the statistical methodology or the 
requirements of the test data appropriate for use in such an accountability system. I have been 
working for more than twenty years to develop a process that will enable a fair, objective measure 
of the impact of districts, schools and teachers on the rate of academic progress of populations of 
students utilizing student achievement test data. From this work and experiences with millions of 
student records obtained from districts within many states, I offer the following as the criteria 
necessary for a robust value-added assessment system. 

Test data requirements for a robust value-added assessment system 

1. The data must come from tests that are highly correlated with curricular objectives. Of 
course, it is desirable to have the correlation between the test and the curriculum to be as 
high as possible. However, since a multivariate longitudinal array of information is used 
to provide the estimates of schooling effects, these correlations do not have to be perfect. 
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2. The scales from these tests must have sufficient stretch to measure the progress of both 
very low and very high achieving students. This has not been observed to be a major 
problem except for a few cases with criterion reference tests that were very narrowly 
focused on mid-level grade skills. 

3. The tests must have appropriate reliabilities. This is usually not a problem for tests with 
40+ items per test. 

Statistical criteria for a robust value-added assessment system 

1. The system must have the capability of providing a multivariate, longitudinal analysis 
using all test data for each student— no matter how sparse or complete. Since there is 
much measurement error around each student’s test score, it is through the exploitation of 
the covariance structure over grades and subjects that the impact, of educational entities on 
rates of student progress, can be measured with the greatest precision. The statistical 
methodology employed for the multivariate, longitudinal analysis must have the capability 
to produce the desired effects without losing the data for individual students due to 
incompleteness of their records. 

2. The system must have the capability of producing best linear unbiased predictors (BLUP, 
or other appropriate shrinkage estimates) of the effects of schools and classrooms on the 
rate of academic progress of student populations. 

3 . If the effects of teachers on student progress are to be estimated, the system must have the 
capacity to appropriately handle team teaching and departmentalized instruction as well as 
self-contained classrooms. 

4. The system needs to have the capacity to use test data from a diversity of sources without 
requiring an equating of previously administered tests. 

Additional diagnostic opportunities from the longitudinally merged data are available . Beyond the 
use of a longitudinally merged database to provide input into a value-added analytical process to 
produce measures for accountability purposes, the availability of this type of database presents 
several opportunities for the extraction of positive diagnostic information to be available to 
practitioners that heretofore could not be provided. In appropriate ways, for each district and 
school (and classroom if the proper identification is available on each student’s test record) the 
rate of progress for low, mid and high achieving students can be measured and reported. This 
type of reporting allows educators a disaggregated view of student progress so that they can 
determine whether the existing combination of curriculum and instructional practice adequately 
provides opportunities for appropriate growth for their students at various levels of previous 
achievement. . 

Once these databases have been developed, the opportunity to use the test data in more proactive 
ways can be exploited. In our most recent efforts, we are providing educators projections for 
individual students which show the probability of a student obtaining various levels of 
achievement in the future, assuming the specific student stays on the academic trajectory indicated 
by past performance. This information will inform educators as to which students may perhaps 
need a different curricular pathway or special resources to change the trajectory, thus increasing 
the probability of achieving the desired endpoints in the future. 
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The NCLB legislation will provide the testing infrastructure to allow educators to more effectively 
manage the progress opportunities for many students who are presently underserved. Analyzing 
the data available with more robust statistical procedures than the AYP reporting requires will 
offer educators precise and reliable information to guide their decisions, allowing them to mitigate 
the unintended consequences of the legislation. Students in states that add a value-added 
dimension to their state accountability system will have greater protection from educational 
practices that result in inequitable student opportunities. 
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